In filmmaking (either virtual, such as in an animated film, or real, such as in a live-action film), when an operator uses a control system to manipulate objects such as a camera, light, character, or similar object, the control system defines a coordinate system (or reference frame) for manipulating the position and orientation of the object. The choice of reference frame has significant impact on the cognitive effort required on the part of the operator to perform a desired movement.
Consider, for example, the common two dimensional coordinate systems known as a Cartesian coordinate system and a polar coordinate system. An example of a device for drawing that is controlled by a Cartesian coordinate system is the well known children's toy Etch-a-Sketch™, which has two dials, one for moving a cursor in a vertical direction and the other for moving a cursor in a horizontal direction. An example of a device for drawing that is controlled by a polar coordinate system is an adjustable drafting compass.
Notice that drawing a square using the Etch-a-Sketch™ toy is trivial, but that drawing a circle with this toy is exceedingly difficult (if not impossible) for a human operator. Although perhaps a computerized input control could cause a decent circle to be drawn, the point is that a human operator would find this difficult. By contrast, by using an adjustable drafting compass, a human operator can easily draw a circle, as well as other movements attuned to polar coordinates. Notably, the square that was trivial in the Cartesian coordinate system would be exceedingly difficult to draw using the compass.
Actually, a more appropriate device for polar movements is a drafting table with a hold in the drafting surface through which a string passes. If the string is attached to a pen and can be pulled from below the table, changes along a radial axis with constant angle can be done by pulling or releasing the string, while movements along an angular axis with constant radius can be made by moving the peen while keeping the string taut.
Naturally, in order to use this device to easily draw a circle with a different center, the fixed point of the string (if not a hold) must move over the drafting surface. Likewise, in order to draw a rotated square, the Cartesian coordinate system should rotate. In general, the ease of controlling movement or transitions in a space can depend on the type of coordinate system and the orientation (e.g., origins, axis directions, etc.) and on the ability to change orientations.
It is instructive to consider the problem related to cameras. Common translation coordinate systems used for camera control individually include camera relative coordinates, world relative coordinates, and crane relative coordinates. For example, in first-person video games, the most common coordinate system is camera relative coordinates. The scene portrayed in the game typically is a camera that moves with a controlled character in the game.
On a scene, such as a stage, world relative coordinates can be used. The user moves the camera relative to a reference point on the stage to obtain the desired view of an action on the scene.
Crane relative coordinates are typically associated with skilled motion picture camera operators. Specifically, by manipulating a crane with a top mounted camera, the best view of a scene may be taken and maintained.
In many cases, the choice of coordinates system has a major impact on the cognitive effort required by the programmer or artist in order to perform a desired object movement, whether this movement is real or virtual. This cognitive effort is particularly relevant to optical transceivers, such as cameras and light sources.
Simply stated, in the case of a camera, the operator must concentrate on both the composition of images being recorded and the path of movement of the camera. Two common examples can illustrate this phenomenon.
Camera relative translation coordinates greatly complicate the task of moving the camera along an arbitrary (straight) line path that is not coincident to the axis of the camera. This is especially true if the camera is panning to aim at a fixed target while moving along the arbitrary (straight) line path. On the other hand, world relative translation coordinates greatly simplify the same task—the direction of movement is defined using the world relative coordinates. When using world relative coordinates, then, during the movement, the camera operator can keep the camera panned at the fixed target (generally through the use of an additional panning control).
In contrast, world relative coordinates complicate the task of following behind an arbitrarily moving target where the camera is aimed. For example, assume that a character has motion across a scene in an action that is not coincident to any of the world relative coordinates and constantly and irregularly changes motion relative to those world relative coordinates. Using world relative coordinates, following the character is difficult at best. On the other hand, camera relative coordinates greatly simplify the same task. All the camera operator has to do is follow the character with reference to the camera image.
Certain other coordinates systems, such as “Crane Axes” may constrain the operator to performing a class of movements which is difficult to describe in Cartesian coordinates but which is stylistically familiar to viewers of a certain type of movie or television show. Camera operators spend many years learning these motions; they become second nature.
It is to be emphasized that this is not a case of a system being user friendly. It is rather a case of making the system adaptable to that particular coordinate system in which the operator is capable of conceiving with conservation of cognitive effort. In many cases, even with the most experienced of camera operators, it is an impossible complication for those camera operators to operate in camera movement coordinate systems that are not compatible with their “on the set” current thinking.
Definition of Terms
Certain terms, as used in this disclosure, have the following defined meanings:
Object. Something that either moves or can be moved relative to a reference coordinate system. The word includes optical transceivers, cameras, lights, characters, and props capable of movement or being moved relative to a reference coordinate system.
Optical transceiver. A system that either receives or transmits light. It includes a light receiving implement such as a camera, either real or virtual, which typically receives and/or projects light along an axis.
Camera. A device that converts light from real world or virtual images into either chemically collected images and/or electrically collected images for animation or real world recorded images. Camera is not intended to restrict the domain of these claims to real or virtual cameras. In particular, this term applies to cameras and other viewing applications in the control of characters, avatars, or other objects in video games.
Light. A device that projects light on to real world or virtual characters, props, or scenes for providing illumination to the ultimately produced and collected images. Typically, the light has a source, a solid angle of propagation, intensity, and color.
Object relative coordinates. These coordinates can be optical transceiver relative, camera relative, light relative, or object relative coordinates: These systems provide controls which move “Forward/Back,” “Right/Left,” and “Up/Down,” all defined relative to a current orientation. In the case of the optical transceiver, camera, or light, the current orientation is the axis of light transmission or reception. In the case of an object, the current orientation is usually the “front” or normal direction of motion. For example, this would be the bow in the case of a ship, the head in the case of a human or animal, etc.
World relative coordinates. These systems provide controls which move “North/South,” “East/West,” and “Up/Down,” all defined using directions external and independent of orientation. Within the category of “World” coordinate systems, it is possible to define an infinite number of “World” reference frames which differ only in the orientation of the coordinate axes. Using the “North/South/East/West” terminology, it is possible to define “North” using the magnetic North pole of the Earth, using the spin axis of the Earth, using the major axis of a rectangular room, and/or using any other definition which is convenient for the system or operator, even if it does not align with any conventional definition of “North.”
Crane Axes coordinates (or Forward Kinematic coordinates). These systems provide controls which rotate individual joints in a segmented crane or robotic arm (thereby moving the end of the crane or arm in some way).
Motorized Track. These systems provide controls which move along a physically constructed (and often curved) track.
Rotation Coordinate Systems. The variety of rotation coordinate systems used for causing rotation space relative to an axis.
Model. A three-dimensional shape, usually described in terms of coordinates and mathematical data, describing the shape of any character or object. Examples of characters include actors, animals, or other beings whose animation can tell or portray the story. In the usual case, the model is typically provided in a neutral pose (known in the art as a “da Vinci pose”), in which the model is shown standing with limbs spread apart and head looking forward. It is understood in the art that in, many situations, the generation of the model can be extraordinarily expensive. In some cases, the model is generated, scanned or otherwise digitized with recorded spatial coordinates of numerous points on its surface. A virtual representation of the model can occur when the data is reconstructed. Furthermore, the model may include connectivity data, such that the collection of points defining the model can be treated as the vertices of polygonal approximations of the surface shape of the model. The model may include various mathematical smoothing and/or interpolation algorithms. Such models can include collections of spatial points ranging from hundreds of points to hundreds of thousands or more points.
Render. To make a model viewable as an image, such as by applying textures to a model and/or imaging the model using a real or virtual camera or by photographing a real object.
Rig. In general, the term “rig” is used to refer to a physics engine that specifies how movement of the model should translate into animation of a character based on the model. This is the software and data used to deform or transform the “neutral pose” of the model into a specific “active pose” variation of the model. Taking the example of the human figure, the rig would impart to the model the skeletal joint movement including shoulder, elbow, hand, finger, neck, head, hip, knee, and foot movement by way of example. By having animation software manipulate a rig incorporated to a model, animated movement of the model is achieved.
Texture. In the usual modern case, “texture coordinates” are mapped onto the surface of a model to provide a digital image portrayed by the model as manipulated by the rig.
Virtual Character. The model as deformed by the rig and presented by the texture in animation.
Virtual Set. The vicinity or fiducial reference point and coordinate system with respect to which the location of any element may be specified.
Prop. An object on the virtual set usually comprising a model without a rig.
Scene. A virtual set, one or more props, and one or more virtual characters.
Action. Animation associated with a scene. It should be noted that upon editing of the final animation story, portions of an action may be distributed without regard to time for example at the beginning, middle and end of the animation story.
Editing. The process by which portions of actions are assembled to construct a story, narrative, or other product.
Actuator. A device such as a mouse or keyboard on a personal computer enabling input to the animation software. This term includes our novel adaptation of a “game controller” for imparting animation to characters.